Training models that generalize to new domains at test time is a problem of fundamental importance in machine learning.

Neural Information Processing Systems http://nips.cc/

Even if the optimal policy is obtained precisely, it is often the case the optimal policy is deterministic.

Wethus establish that policy iteration on reward-robust MDPs can have the same time complexityasonregularizedMDPs.

Whiletheseapproaches arewidely used inpractice andachieveimpressiveempirical gains, their theoretical understanding largely lags behind. Towards bridging this gap, we present a unifying perspectivewhere several such approaches can beviewed asimposing a regularization on the representation via alearnable function using unlabeled data. Wepropose adiscriminativetheoretical framework for analyzing the sample complexity of these approaches, which generalizes the framework of [3] to allow learnable regularization functions.

Training models that generalize to new domains at test time is a problem of fundamental importance in machine learning.

Existing machine learning algorithms including deep neural networks achieve good performance in cases where the training and the test data are sampled from the same distribution. While this is a reasonable assumption to make, it might not hold true in practice.

We consider regularized empirical risk minimization problems. In particular, we minimize the sum of a smooth empirical risk function and a nonsmooth regularization function. When the regularization function is block separable, we can solve the minimization problems in a randomized block coordinate descent (RBCD) manner. Existing RBCD methods usually decrease the objective value by exploiting the partial gradient of a randomly selected block of coordinates in each iteration. Thus they need all data to be accessible so that the partial gradient of the block gradient can be exactly obtained.

The conditional average treatment effect (CATE) is widely used in personalized medicine to inform therapeutic decisions. However, state-of-the-art methods for CATE estimation (so-called meta-learners) often perform poorly in the presence of low overlap. In this work, we introduce a new approach to tackle this issue and improve the performance of existing meta-learners in the low-overlap regions. Specifically, we introduce Overlap-Adaptive Regularization (OAR) that regularizes target models proportionally to overlap weights so that, informally, the regularization is higher in regions with low overlap. To the best of our knowledge, our OAR is the first approach to leverage overlap weights in the regularization terms of the meta-learners. Our OAR approach is flexible and works with any existing CATE meta-learner: we demonstrate how OAR can be applied to both parametric and non-parametric second-stage models. Furthermore, we propose debiased versions of our OAR that preserve the Neyman-orthogonality of existing meta-learners and thus ensure more robust inference. Through a series of (semi-)synthetic experiments, we demonstrate that our OAR significantly improves CATE estimation in low-overlap settings in comparison to constant regularization.